Anytime, Anywhere Anomaly Recovery through an Online Robot Introspection Framework

Robotic introspection and online decision making have been an area of increased focus. The goal is to endow robots with the ability to understand their actions and make timely decisions to reach their goals. Particularly, in unstructured environments, external perturbations are hard to model in low-level control systems and often lead to failure. Robots must then understand nominal and anomalous conditions and trigger timely responses to behaviors that allow the robot to recover and even learn from them and prevent them. Our contribution is the implementation of a fast and robust robot introspection system that allows recovery from (one or multiple) anomalous situations at any point in the task. The system handles both internal modeling errors as well as external perturbations. The robustness of the system is demonstrated across multiple manipulation tasks. The system assumes tasks are decomposed into a sequence of nodes, where each node performs a dual role: one of motion generation and one of introspection. Motion generation is flexible and can be done with any type of accessible approach. Introspection is done by modeling the robots multimodal signals using a range of HMMs including nonparametric Bayesian hidden Markov models. Such models yield strong expressive power to discriminate both nominal and anomalous situations. We made use of a generic strategy for recovery that is easy and flexible to design across different tasks. A new metric for anomaly detection, critical in the proper assessment of the system after recovery has taken place was also designed. We show how the system recovers from both pose estimation errors that lead to collisions in pick tasks as well as external human collisions (which are more likely in co-bot relatedtasks). Furthermore, the system is able to robustly recover from collisions that occur at multiple points in the task; even, when anomalies repeatedly appear at a specific point in the task. Supplemental information including videos, code, and result analysis can be found at [1].